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Interest in online learning (OL) has grown rapidly and will continue to do so (Bonk, 2002; Sofres, 2001). 

An increasing number of universities are offering OL opportunities (Duffy & Kirkley, 2004). A number of studies 
have discussed and linked online collaboration through group discussions with better opportunities to promote 
quantity and quality of student interaction, engagement, satisfaction, and higher-order learning (Hiltz, Coppola, 
Rotter, & Turoff, 2000; Mikulecky, 1998; Nachmias et al., 2000). Nevertheless, it is common to see little discussion 
among online groups that work collaboratively. Participants tend to post without replying to other participants’ 
contributions, and they remain in their comfort zones (Collett, Kanuka, Blanchette, & Goodale, 1999). Threaded 
discussions often end in trivialized conversations in which frequently several members remain as passive observers 
(Klenrm & Snell, 1996). As Kanuka & Garrison (2004) point out “To date there have been few empirical studies on 
the use of asynchronous text -based Internet communication technologies and their ability and/or effectiveness to 
facilitate higher levels of learning’’ (p.31). OL might have great potential to promote critical thinking, however, to 
implement OL, and especially online discussions in a way that actualize this potential has proved to be a real 
challenge. 

One of the main arguments in studies that have linked online collaboration through group discussions with 
better learning opportunities is that group discourse serves as a core in increasing individual critical thinking within 
OL environments (Angeli & Bonk, 2003; Garrison, Anderson, & Archer, 2001; Jeong, 2001). Critical thinking is 
defined as “the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, 
synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, 
reasoning or communication, as a guide to belief and action (Paul & Elder, 2001).’’ Duffy, Dueber, & Hawley 
(1998) place inquiry at the heart of learner-centered online environments, and associate critical thinking with an 
inquiry process in which a learner inductively or abductively solves his puzzlement through hypothesis generation, 
data gathering and evaluation, considering alternatives, and resolving for a rational solution. 

While critical thinking tends to be defined as an individual cognitive process, Garrison, Anderson, & 

Archer (2000) developed a “community of inquiry’’ model that provides a conceptual framework to study learning in 
online environments, placing critical thinking in the context of collaborative work. In this model they use the term 
“cognitive presence’’ to refer to “the extent to which learners are able to construct and confirm meaning through 
sustained discourse in a critical community of inquiry (Garrison et al., 2001),” focusing in this way on the sustained 
interactions that help individuals to develop critical thinking as a process and outcome (Archer, Garrison, Anderson, 
& Rourke, 2001; Garrison et al., 2000). 

In addition to cognitive presence, Garrison et al. (2000) also emphasize the importance of social presence 
in their community on inquiry model. Social presence is the sense of relationship that can arise among people 
participating in online environments, because participants project their personal characteristics into the community 
of inquiry, thereby presenting themselves as real people (Rourke, Anderson, Garrison, & Archer, 2001). In OL 
social presence can be mediated using text based communication, and it appears to be relevant to promote student 
satisfaction and engagement (Richardson & Swan, 2003). Nonetheless, Wise, Chang, Duffy, &del Valle (2004) 
found that only a basic threshold of social presence is required in orderto support online learners. 

The role of online facilitators who cultivate learners’ cognitive presence has been considered as an 
important factor in promoting students’ critical thinking (Anderson, Rourke, Garrison, & Archer, 2001; Archer et 
al., 2001; Collison, Elbaum, Haavind, & Tinker, 2000; Richardson & Swan, 2003). Thus, identifying guidelines for 
facilitating cognitive presence in online settings is an essential issue in order to encourage learners to think critically 
in a collaborative environment. Collison et al. (2000) identified several facilitation strategies to promote critical 
thinking in online environments. These strategies suggest that facilitators should promote cognitive presence in two 
stages: in the first stage, the facilitator uses techniques to help learners clarify their ideas, and come to a common 
understanding in order to “sharpen the focus”. In the second stage, the facilitator supports learners to deepen their 
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thoughts and dialogue by articulating more in-depth discussions on the common ground. In this context the 
facilitator is not acting as a content expert but as an experienced learner and mentor that promotes students critical 
thinking and knowledge construction, mainly by asking questions that will further push their thinking and inquiry. 

The purpose of this exploratory experimental study was to examine how online cognitive facilitation that 
promotes cognitive presence, while keeping facilitator’s social presence constant, affects student learning and 
satisfaction in an online community of inquiry. In this context, we explored the following questions: 

1) Does online high cognitive facilitation promote collaboration and critical thinking? 

2) Does online high cognitive facilitation promote student learning and improve satisfaction? 

Methodology 

Participants 

Participants were 12 graduate students studying in the Instructional Technology Department of a large 
Midwestern university, with 1 1 of them enrolled in the distance education (DEJ masters program, and one of them a 
residential student. Participants were equally distributed in terms of gender (50% male, 50%female). Two 
participants were in the age range 20-30, three in the range 31-40, five in the age range 41-50, and two of them were 
over 5 1 . 

The participants were randomly assigned to four 3-person groups. Three of the participants dropped from 
the study after the first days of discussion, which caused 3 groups to continue as 2-person groups. Additionally, two 
participants who were active during the discussion, for unknown reasons did not complete the online final 
instruments. 

Task 

During the study, students were involved in an 1 1-day online asynchronous discussion about the use of DE 
in K-12 environments. The whole discussion period lasted 13 days, but there was a 2-days break within that period 
due to the Thanksgiving holiday. To facilitate the discussion each group was asked to identify the five most 
important issues that should be considered in determining the role of DE in K- 12 education. They were to think of 
their team as consulting to state departments of education on the use of DE. Participants were asked to post the 
issues they agreed on and the rationale for each of them by the end of the discussion. Four brief magazine articles on 
the K-12 DE topic were selected to serve as resources for the discussion. One of these articles was given to all 
members in each of the groups. The other three articles were distributed randomly among the members of each 
group in order to minimize the workload of the participants, and to foster the diversity of ideas in the discussion. 

The participants were instmcted not to look for more articles on the topic, not to share articles with their teammates, 
and not to discuss the topic with other members of their class. They were to make at least one posting everyday, and 
avoid using email except in emergency or to communicate personal information to the facilitator. 

Design 

The independent variable was the degree of cognitive facilitation provided by the facilitator, with two 
groups receiving high cognitive facilitation and two groups receiving low cognitive facilitation. 

There were two facilitation teams each of which consisted of three people from the research team. Each 
team was responsible for the facilitation of one group in the high cognitive facilitation condition, and another group 
in the low cognitive facilitation condition. One person from each team took the lead in generating and posting 
messages , and the others gave feedback before the messages were posted. Facilitators were to post a message every 
day. 

In both high and low conditions, facilitators avoided using didactic teaching strategies, acting as content 
experts, and introducing new material. Both facilitators included high social presence in their postings. For the 
purpose of this study facilitator high social presence is defined as a style of communication that makes active use of 
written expressions to compensate for the lost non-verbal cues in computer mediated communication (Rice & Love, 
1987). The intentional use of social presence cues (Swan, 2003; Rourke, Anderson, Garrison, & Archer, 2001; 
Abdullah, 1999) should allow facilitators to be perceived as friendly within the mediated communication. The cues 
used were humor/playful asides, expression of emotion, self -disclosure, addressing people by name, 
greetings/phatics, and allusions of physical presence. 

In the high cognitive facilitation condition the overall goal of the facilitators was to promote critical 
dialogue and evidence based argumentation among team members without providing direction or points of view on 
the specific content. Cognitive facilitation included inquiry learning facilitation techniques such as asking for the 
clarification and elaboration of the presented ideas, promotion of the use of the articles, and the sorting and synthesis 
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of ideas. In the low cognitive facilitation condition facilitators did not use any of these strategies but rather 
encouraged the students to discuss and share their ideas. 

Facilitators were trained in responding with high and low cognitive facilitation messages. Throughout the 
facilitation process they used two facilitation guides prepared by the research team; one for each condition. The high 
cognitive facilitation guide (Table 1) included specific strategies and detailed examples for each one. The low 
cognitive facilitation guide focused mainly on social and motivational presence, and instructed facilitators to avoid 
using any of the strategies used on the high cognitive facilitation. The strategies on the high cognitive facilitation 
guide (Table 1) were drawn from the model defined by Collison et al. (2000). 

Table 1: Cognitive Facilitation Strategies* (adapted from Collison et al., 2000) 

General Strategy Category Specific Strategies 

IDENTIFYING DIRECTION: Selectively highlight or paraphrase relevant 
discussion items in order to refocus or redirect conversation, possibly 
weaving several discussion threads or ideas to provide a new focus. 

SORTING IDEAS FOR RELEVANCE: Ask students to classify or form 
comparisons. Alternatively, if needed you can call attention to sorting of 
ideas, making public the sorting mechanism, to focus on relevance and 
importance. Highlight ‘hidden gems’ in postings to bring them out of 
obscurity. 

FOCUSING ON KEY POINTS: Ask them to summarize or synthesize. If 
necessary eventually you can highlight key contributions, essential concepts, 
and connections so far. Bring to light potential gaps or tensions. Provide big 
picture, but do not summarize in detail or infer future directions - push 
participants to draw these inferences and assessments by themselves. Ask 
them to evaluate the strength on their ideas, seek judgments and assessments, 
and eventually reach consensus. 

FULL-SPECTRUM QUESTIONING: Use a wide questioning approach 
(Who? What? When? Where? Why?) to push participants to examine their 
own personal, or collective, thoughts and beliefs. 

■ Push them to go beyond by asking “so what?’’ in a specific context 

■ Ask them to consider other perspectives that they may not have 
thought about before 

■ Ask them to clarify or elaborate of their ideas 

■ Ask them explore their assumptions and sources, and provide a 
rationale or examples for their ideas 

■ Ask questions to identify cause and effects/outcomes 

■ Ask the team to solve discrepancies in the ideas 

■ Ask questions considering appropriate action or inquiry especially if 
the discussion is stuck 

MAKING CONNECTIONS: Stretch the participant’s imagination or 
conceptual frames to consider obscure but essential similarities. Moving 
beyond the barriers of previously held beliefs or assumptions that may block 
these connections across contexts or at deeper levels. 

HONORING MULTIPLE PERSPECTIVES: Lay out the landscape of 
different views present on the discussion. This is usually the last stage before 
the group completes their final task. 

* Note: examples included on the facilitation guide are not shown on this table 

Instruments 

All of the instruments used in the study were administered online. They included a pre -survey, a post- 
survey, and two final tests. 

Pre-survey. The first section of the pre -survey consisted of two demographic items: name and age. The 
second section included 10 five point Likert -type items where 1 “Strongly Agree’’ and 5 “Strongly disagree’’, that 



SHARPENING THE FOCUS 

These strategies seek to clarify 
ideas and find common 
understanding among 
participants. Highlight relevant 
ideas and key contributions, 
bringing coherence and pushing 
the dialogue forward. 



DEEPENING THE DIALOGUE 

Build on common ground by 
promoting that participants 
examine their own beliefs and 
assumptions, reflecting on 
perturbations to build new levels 
of understanding. 
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were designed to assess participants' attitude towards the use of DE (three items), their interest in the topic (four 
items) and their attitude toward current state of K- 12 education (three items). Although the internal consistency for 
questions on the three topics was not high enough to validate the existence of scales, individual items still provide 
useful information about the three indicated aspects. The last section consisted of 6 five point Likert -type items 
designed to assess the participants’ level of trust. The trust items were drawn from Yamagishi & Yamagishi’s (1994) 
scale of general trust, reaching in this study an internal consistency of .92. 

Post-survey. The post-survey consisted of 31 five-point Likert -type items where 1 “Strongly Agree” and 5 
“Strongly disagree”. Six variables were measured. The first three scales were designed to assess the success of the 
treatment, and measured participants perceived cognitive facilitation (nine items with an internal consistency of .87), 
perceived facilitator social presence (seven items with an internal consistency of .89), and perceived facilitator 
motivational presence (three items with an internal consistency of .87). The last three scales assessed overall 
perceptions of participants’ experience during the discussion, and measured overall satisfaction (six items with an 
internal consistency of .8), perceived learning (three items with an internal consistency of .79), and relevance/use of 
the articles (3 items with an internal consistency of .57). 

Independent samples t tests were used to compare the results of the initial and final surveys reliable scales. 

Recall Test. A recall test prompted the participants torememberas much as ideas they could from the 
articles they read and from their group discussion. This test was used as a basic measure of learning. Participants 
were instructed not to go back to the articles or the discussion forum before or during the recall task, and they were 
to use at most 20 minutes to complete the test and to report the time they used to complete it. 

Open-ended Test. An open-ended test was used as a more elaborated measure of learning. It was a short test 
including four open-ended questions focusing on student’s understanding of some of the basic issues included across 
all K-12 DE articles that were read. 

Procedure 

The discussion took place on the SiteScape asynchronous discussion forum (SSF). Because the participants 
had been using the same forum for their class discussions, they were familiar with the interface. 

One day before the discussion activity began all participants were sent an email message including a link to 
the online pre -survey, and an attached file including the task instructions. The task instructions were also available 
on SSF. 

Once all the pre-survey responses were received, facilitators posted their first message on SSF. Two 
different welcome messages were composed as the first facilitators’ posting, one for the low cognitive facilitation, 
and one for the high cognitive facilitation condition. In addition to posting the welcome message on SSF, the same 
message was sent to participants by email in order to let them know that the discussion had started. After the 
welcome message, each facilitator made up their own messages according to the flow of the discussion, and 
according to the experimental condition of the groups. Due to lack of response from some participants, both 
facilitators did not post any message in a few occasions towards the end of the study in order not to dominate the 
discussion. Facilitators used email only as a reminder to those who were not actively participating. In these emails 
students were instructed not to reply but to post their contribution to SSF. 

Two days after the discussion activity ended, all participants received an email message including a link to 
the post-survey, and another link to the recall and the open-ended tests. 

Analysis 

Content Analysis. Garrison et al.’s (2001) content analysis categories were used to assess the degree of 
critical thinking exhibited in the online discussion. These categories divide critical thinking in four phases: 
triggering event , exploration , integration , and resolution. Archer et al. (2001) define these phases as the idealized 
logical sequence of the process of critical inquiry. The first phase, triggering event, is an initiation phase of inquiry 
in which an issue or problem is raised. In the exploration phase participants’ brain-storming, questioning, and 
exchange of information takes place. Moving beyond exploration learners come to the integration phase which is an 
iterative effort to construct shared meaning within the community of inquiry. The fourth phase is the resolution of 
brought up in the triggering event. The data that did not belong to any of these categories were put under a fifth 
category called other. 

In addition to critical thinking two other variables were analyzed: collaborativeness of the participants 
during discussion and the source of support they used as evidence for their arguments. Collaborativeness was 
defined as participants’ referring to the arguments of their teammates in a capacity to extend their contributions. 
Collaborativeness was coded into three categories depending on the reference to other participants: no reference to 
others, referring to one posting of a teammate, and referring to two or more postings of teammates. The source of 
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support was divided into two categories according to the types of evidences used in the arguments: support by 
referencing the readings, and support by referencing personal experience. 

Two coders were responsible for the content analysis of the discussions. Sentence-level idea units for 
assessment of critical thinking and message-level idea units for assessment of collaborativeness and support were 
used for being the most appropriate in terms of usefulness and reliability. Coders worked on a small sub-set of data 
to develop the rules to define the analysis units and code them. After there was agreement on the definitions and 
rubrics of these idea unit analyses, the coders also practiced assigning data to these categories. 

Since one of the coders was also a facilitator, and in order to avoid possible bias, before analyzing the 
discussion postings a third person removed the facilitators’ messages, changed participants’ names in each message 
to a pseudonym, and altered the order of the messages. The two trained coders broke the data into idea units 
individually and then compared their results. The level of agreement for this initial unitization was 96% using 
Holsti’s Coefficient of Reliability measure. In all, the coders agreed on 852 sentence units within the 72 posted 
messages. After resolving discrepancies between the unitization results through discussion, coders individually 
coded a fifty percent sample of the messages into the critical thinking, collaborativeness, and support categories, 
according to the established rubrics. Results of coding were then combined and contrasted. Inter-rater reliability for 
content analysis was not as high as expected and therefore average scores from the two coders’ results were used for 
analysis purposes. 

Test Analysis. To assess participants’ learning, the answers for the recall and the open-ended test were 
analyzed. The results of the recall test were scored by two different raters. Using a set of rules defined during 
practice sessions, raters compared the results to a predefined set of 142 possible items to recall that was extracted 
from all the articles used during the discussions. Raters reached a 75.5% of agreement on their scoring results. The 
results of the open-ended test were also scored by two different raters using a three points rubric with two criteria for 
each question, reaching a 78.9% of agreement. Independent samples t tests were used to compare the results of the 
final tests. 



Results 

Participants’ Incoming Variables 

Results on the initial online survey (with a 5 point Likert scale where l=“Strongly Agree” and 5=‘‘Strongly 
Disagree”), indicate that overall participants’ (n=12) trust levels were high (M=1.61, SD=0.42), excluding one 
outlier (2.5 standard deviations from the sample mean) that showed low trust levels (M=3.5) according to the 
instrument. No significant differences were found between participants on the low and high groups on this variable. 
This result is relevant considering the importance of trust in student participation in online discussions (Yamagishi, 
2001), and indicates that any possible effects were independent of participants’ trust levels. 

All participants also showed a positive attitude toward DE. When asked if they agree with the idea that 
‘‘Distance education is a strategy that will provide real educational opportunities world wide,” eight chose the 
‘‘Strongly Agree” option and four ‘‘Agree” (M=1.3, SD= 0.5). When asked for their interest in the discussion topic, 
again all participants showed high levels of interest. To the statement, ‘‘The use of DE is a topic that interests me”, 
all but one participant answered, ‘‘Strongly Agree,” and the remaining answered ‘‘Agree” (M=l.l, SD=0.3). Finally, 
when asked about their attitude toward K- 12 education, when asked to react to the following statement ‘‘We are 
doing all we can to provide the best education we can in the US,” again participants reacted similarly, in this case 
disagreeing with the statement (M=4.1, SD=0.8). 

Overall these results indicate that all participants were in similar conditions to participate in the discussion. 
They were all interested in the topic; they valued the potential of DE and had some concerns about the current state 
of K-12 education. All these are indicators that they could have a live discussion on the proposed topic: the use of 
DE in K-12 settings. 

Participants’ Perception of the Treatment 

The three final survey scales (Table 1) that assessed how successful the treatment was, indicate no 
significant differences between conditions for their perceived facilitator social and motivational presence. 
Nonetheless, despite of the small sample size a significant difference t(5) = 3. 17, p = .025 was found for the results 
of the perceived cognitive facilitation scale. These results indicate that the intended manipulation was successful and 
perceived by the participants. They should all have the same levels of social and motivational presence but differ in 
the level of cognitive facilitation. 
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Table 1: Treatment perceptions for high and low cognitive facilitation conditions (5 point Likert scale 1 =SA and 
5=SD, N=7) 



Condition 


Social Presence 


Motivational Presence 


Cognitive Facilitation 


Mean 


1.52 


2.00 


2.15* 


High 


SD 


0.36 


1.20 


0.65 


Mean 


1.75 


2.00 


3.36 


Low 


SD 


0.88 


1.05 


0.37 



* Significant at the .05 level 



Discussions Analysis 

In the context of an exploratory study, content analysis results serve the main purpose of revealing patterns 
and trends that should inform thinking for subsequent studies. Thus, rather than reporting a detailed statistical 
analysis we carried out the content analysis focusing on conceptual indicators. 

In terms of critical thinking, several patterns emerged indicating differences between treatments. Table 2 
shows combined analysis by averaging the coders’ ratings and the two groups on each condition. Results indicate a 
considerably higher percentage of critical thinking by sentence units in the high cognitive facilitation groups. 



Table 2: Percentage of idea units by critical thinking category. Combined coders, combined groups 



Content of idea 
units 


Low cognitive facilitation groups 
(n=178 units)* 


High cognitive facilitation groups 
(n=248 units) 


Other 


33.99% 


12.40% 


Trigger 


5.90% 


2.72% 


Exploration 


35.96% 


39.72% 


Integration 


17.42% 


42.34% 


Resolution 


6.74% 


2.82% 



* n = mean number of idea units at the sentence level for combined treatment groups 

Regarding collaborativeness, results show similar levels of collaboration between treatments (Table 3), 
with a slight tendency in the high condition for having in total more references to other participants’ messages. 

Table 3: Number of messages by level of collaborativeness - Combined coders, combined groups. 

Number of prior messages Low cognitive facilitation groups High cognitive facilitation groups 
referenced (n=41 units)* (n=31 units) 

None 62,20% 59.68% 

One 28.05% 33.87% 

Two or more 9.76% 6.45% 

* n = mean number of idea units at the message level for combined treatment groups 

Regarding the use of resources to support discussion postings (Table 4), there was an important difference 
between high and low groups, with high cognitive facilitation groups providing article based support to their 
postings more than twice those on the low cognitive facilitation groups. 
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Table 4: Number of messages by article based support - Combined coders, combined groups. 



Support on 


Low cognitive facilitation groups 


High cognitive facilitation groups 


articles data 


(n=41)* 


(n=31) 



Yes 12.20% 27.42% 

No 87.80% 72.58% 

* n = Mean number of idea units at the message level for combined treatment groups 

Personal experience support was a coding decision made based on whether messages showed evidence of 
including examples from the participant’s experience with the intention of using them to support an argument. Table 
5 presents combined coder and treatment group results for personal experience support, showing very little 
difference between the high and low cognitive facilitation groups, both exhibiting personal experience support in 
nearly 30% of the messages. 

Table 5: Number of messages by personal experience support- Combined Coders, Combined Treatment Groups 



Personal Experience 


Low cognitive facilitation groups 


High cognitive facilitation groups 


Support 


(n=41)* 


(n=31) 


Yes 


30.49% 


29.03% 


No 


69.51% 


70.97% 



* n = mean number of idea units at the message level for combined treatment groups 



Final Survey Results 

The three final survey outcome variables; overall satisfaction, perceived learning, and self-reported use of 
the articles, indicate no significant differences between conditions. Nevertheless, in all cases there is a clear 
tendency for participants on the high cognitive facilitation groups to have higher ratings, as shown on Table 6. They 
were more satisfied with the discussion, their perception of learning was higher, and their self-reported use of the 
articles was also higher. 



Table 6: Fined survey results by facilitation conditions (5 point Likert sccde 1 =SA. 5=SD, n=7) 



Condition 


Overall Satisfaction 


Perceived Learning 


Self reported use of articles 


High 


Mean 


2.00 


2.00 


1.11 




SD 


0.60 


0.33 


0.19 


Low 


Mean 


2.42 


2.42 


1.67 




SD 


0.52 


0.63 


0.38 



Final Tests Results 

In terms of learning outcomes (Table 7), no significant differences were found, but there is a tendency for 
those in the high cognitive facilitation groups to have a higher number of recalled items and a higher score on the 
open ended test. 



Table 7: Learning outcomes by conditions (n=7) 



Condition 


Number of items recalled 


Test Score(24 points scale) 


High 


Mean 


11 


20.38 




SD 


7.35 


2.10 


Low 


Mean 


9.5 


19.13 




SD 


6.03 


1.93 
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Discussion 

The purpose of this pilot study was to explore the relationship between high cognitive facilitation, critical 
thinking, and learning in online collaborative environments. 

According to the results of the initial survey all participants were in similar conditions to participate in the 
discussion. They were interested in the topic, valued the potential of distance education and had some concerns 
about the current state of K-12 education. In addition, all participants (with the exception of an outlier) showed high 
trust levels, which was relevant for the study considering the importance of trust in student participation. All these 
were indicators that the manipulation could work and that participants could have a live discussion on the proposed 
topic. Cognitive facilitation was successfully manipulated and participants in the high condition did perceive their 
facilitator as providing a significantly higher level of cognitive facilitation. Simultaneously, there was no difference 
in the perception of the facilitator as being friendly (high social presence), and providing good motivation for the 
discussion. 

These results are relevant for future studies on cognitive facilitation, because they show that cognitive 
facilitation can be successfully manipulated. The results also indicate that potential differences among conditions 
were not due to different levels of personal interest or trust, neither to different levels of social or motivational 
presence on the facilitator’s side. 

The first research question considered if high cognitive facilitation promoted critical thinking among 
participants or not. Based on the results of our analysis, we do see support for a link between high cognitive 
facilitation and critical thinking. Although our sample size was very small, we did see evidence of more critical 
thinking taking place among participants in the groups receiving high cognitive facilitation than among those 
receiving low cognitive facilitation. While the low group had 66% of critical thinking units, the high group had 88%. 
Specific support to this trend comes from the higher percentage of idea units classified as integration for groups 
receiving high cognitive facilitation. According to Garrisons et al.’s (2003) model, integration is the most important 
category indicating critical thinking process. Low cognitive facilitation groups had a high number of idea units 
classified under other (34%), i.e. social or general postings with no direct relation to critical thinking, and 
concentrated their critical thinking on the two initial levels of the process (trigger and exploration 42%). 

Concerning the additional categories examined, personal experience was problematic due to low inter-rater 
reliability. We think there was a good deal of confusion between the coders in the definition used to code this 
category. This disagreement may also have had an impact in the critical thinking coding as well, since personal 
experience was used as specific indicators in Garrison et al.’s examples (2003). We recommend that clearer 
definition and agreement be reached on the meaning of personal experience. 

While our inter-rater reliability was lower than we would have preferred, these indicators give us hope that 
high cognitive facilitation may positively impact critical thinking in OL, and would therefore be a worthwhile 
practice to foster. 

There were other relevant problems that arose in this study that need to be addressed and considered on 
future studies. The rating cues for critical thinking, based on the Garrison et al.’s model (2003), were not equally 
understood by the two raters, and better agreement could be reached by a clearer definition of the categories. In 
particular, the ambiguity between integration and exploration categories needs to be clarified. These categories 
created the most recognizable differences in coding between the coders. 

One of the coders in this study was also a facilitator for two of the groups. This coder’s familiarity with the 
discussions, and the potential of inferring context from memory, may have had an impact on the coding results, 
particularly his higher count for integration. It is recommended that future coding is not done by one of the study 
facilitators. 

Context is also an important consideration. We “sanitized” the data and changed the order of the messages 
so the coders would not recognize the postings as being from one of the conditions. Nevertheless, the flow of an 
entire discussion thread could have provided clearer indication of the content categories for individual messages. If 
coding is done by non facilitators the original flow of the discussion could be used with no foreseen problem. 

Unitization was a struggle for this study. We originally wanted to code at a “unit of meaning” level, but had 
much difficulty reaching agreement between coders on these units. To reduce ambiguity, we decided to code at the 
sentence level. This created much higher inter-rater reliability in unitization, but introduced the problems of volume 
of units (852 sentences), and lack of context. Garrison et al. (2003) decided to unitize at the message level. This 
makes logical sense as most messages are trying to accomplish one predominant task. Additionally, the model we 
used for analysis was developed and previously applied for coding at the message level, and did not translate ideally 
for applying it at the sentence level. We would recommend future studies use the message level for unitization, or 
develop a better rating scale more suited to the agreed upon units. 

The second research question considered if high cognitive facilitation, and the consequent higher presence 
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of critical thinking among participants, results in higher levels of learning among participants or not. Results 
indicate that there were no statistically significant differences between the learning outcomes of the two conditions, 
or between the effects on perceived learning or satisfaction. Nevertheless, results also suggest a consistent tendency 
in which participants in the high condition showed higher satisfaction, higher perceived learning and higher use of 
the articles. In terms of learning outcomes, no significant differences were found in both the recall and the open 
ended tests, but again there is a tendency for those in the high cognitive facilitation groups to have, in average, a 
higher number o f recalled items , and a higher score on the open ended test. 

The fact that we had a small sample size might have prevented us from finding significant differences 
among conditions. Additionally, the instruments might not have measured learning in a way that would help us 
discriminate enough what participants learned during the discussion, and what came from their prior experience. 
Another factor that could have impeded appropriate discrimination was that each team member in addition to the 
shared article received an extra one, which made the recall test scoring complicated. Perhaps an interesting topic in 
which participants have none or reduced previous experience could have been selected to avoid this problem. Future 
studies should consider these aspects when further investigating online cognitive facilitation and student critical 
thinking in online environments. Nonetheless, as far as we know online cognitive facilitation has not been 
experimentally manipulated before, and results of this study open an interesting avenue for future research on 
facilitation, critical thinking, and collaboration in online learning environments. 
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